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Abstract 

Novel experimental techniques reveal the simultaneous activity of larger and larger numbers of neu- 
{S) rons. As a result there is increasing interest in the structure of cooperative - or correlated - activity 

in neural populations, and in the possible impact of such correlations on the neural code. A fun- 
damental theoretical challenge is to understand how the architecture of network connectivity along 
with the dynamical properties of single cells shape the magnitude and timescale of correlations. We 
provide a general approach to this problem by extending prior techniques based on linear response 
theory. We consider networks of general integrate-and-fire cells with arbitrary architecture, and 
provide explicit expressions for the approximate cross-correlation between constituent cells. These 
correlations depend strongly on the operating point (input mean and variance) of the neurons, even 
when connectivity is fixed. Moreover, the approximations admit an expansion in powers of the ma- 
trices that describe the network architecture. This expansion can be readily interpreted in terms 
of paths between different cells. We apply our results to large excitatory-inhibitory networks, and 
Qpi demonstrate first how precise balance — or lack thereof — between the strengths and timescales of 

excitatory and inhibitory synapses is reflected in the overall correlation structure of the network. 
We then derive explicit expressions for the average correlation structure in randomly connected 
networks. These expressions help to identify the important factors that shape coordinated neural 
activity in such networks. 

On 

q Author summary 

Is neural activity more than the sum of its individual parts? What is the impact of cooperative, 
• • or correlated, spiking among multiple cells? We can start addressing these questions, as rapid ad- 

. ^ vances in experimental techniques allow simultaneous recordings from ever-increasing populations. 

However, we still lack a general understanding of the origin and consequences of the joint activity 
that is revealed. The challenge is compounded by the fact that both the intrinsic dynamics of single 
cells and the correlations among then vary depending on the overall state of the network. Here, we 
develop a toolbox that addresses this issue. Specifically, we show how linear response theory allows 
for the expression of correlations explicitly in terms of the underlying network connectivity and 
known single-cell properties — and that the predictions of this theory accurately match simula- 
tions of a touchstone, nonlinear model in computational neuroscience, the general integrate-and-fire 
cell. Thus, our theory should help unlock the relationship between network architecture, single-cell 
dynamics, and correlated activity in diverse neural circuits. 
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Introduction 



New multielectrode and imaging techniques are revealing the simultaneous activity of neural ensem- 
bles and, in some cases, entire neural populations [lj-|4] . This has thrust upon the computational 
biology community the challenge of characterizing a potentially complex set of interactions — or 
correlations — among pairs and groups of neurons. 

Beyond important and rich challenges for statistical modeling (5J, the emerging data promises 
new perspectives on the neural encoding of information [6j. The structure of correlations in the 
activity of neuronal populations is of central importance in understanding the neural code (7-13 



However, theoretical [9[ |TT|[T4] - [T6] , and empirical studies 17-19 do not provide a consistent set of 
general principles about the impact of correlated activity. This is largely because the presence of 
correlations can either strongly increase or decrease the fidelity of encoded information depending 
on both the structure of correlations across a population and how their impact is assessed. 

A basic mechanistic question underlies the investigation of the role of collective activity in cod- 
ing and signal transmission: How do single-cell dynamics, connection architecture, and synaptic 
dynamics combine to determine patterns of network activity? Systematic answers to this question 
would allow us to predict how empirical data from one class of stimuli will generalize to other 
stimulus classes and recording sites. Moreover, a mechanistic understanding of the origin of corre- 
lations, and knowledge of the patterns we can expect to see under different assumptions about the 
underlying networks, will help resolve recent controversies about the strength and pattern of corre- 



lations in mammalian cortex |l|20 - 22 . Finally, understanding the origin of correlations will inform 



the more ambitious aim of inferring properties of network architecture from observed patterns of 



activity 123-25 



Here, we examine the link between network properties and correlated activity. We develop a 
theoretical framework that accurately predicts the structure of correlated spiking that emerges in 
a widely used model — recurrent networks of general integrate and fire cells. The theory naturally 
captures the role of single cell and synaptic dynamics in shaping the magnitude and timescale of 
spiking correlations. We focus on the exponential integrate and fire model, which has been shown 
to capture membrane and spike responses of cortical neurons [26]; however, the general approach 
we take can be applied to a much broader class of neurons, a point we return to in the Discussion. 



Our approach is based on an extension of linear response theory to networks (25 , 27 . We start 



with a linear approximation of a neuron's response to an input. This approximation can be obtained 



explicitly for many neuron models [28 - 30 , and is directly related to the spike triggered average 31 



The correlation structure of the network is then estimated using an iterative approach. As in prior 
work (32 - 34 , the resulting expressions admit an expansion in terms of paths through the network. 

We apply this theory to networks with precisely balanced inhibition and excitation in the inputs 
to individual cells. In this state individual cells receive a combination of excitatory and inhibitory 
inputs with mean values that largely cancel. We show that, when timescales and strengths of 
excitatory and inhibitory connections are matched, only local interactions between cells contribute 
to correlations. Moreover, our theory allows us to explain how correlations are altered when precise 
tuning balance is broken. In particular, we show how strengthening inhibition may synchronize 
the spiking activity in the network. Finally, we derive results which allow us to gain an intuitive 
understanding of the factors shaping average correlation structure in randomly connected networks 
of neurons. 
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Network model and linear response theory 



Our goal is to understand how the architecture of a network shapes the statistics of its activity. 
We show how correlations between spike trains of cells can be approximated using response char- 
acteristics of individual cells along with information about synaptic dynamics, and the structure of 



the network. We start by briefly reviewing linear response theory of neuronal responses 29 , 35 36 
and then use it to approximate the correlation structure of a network. 

Network model 

To illustrate the results we consider a network of N nonlinear integrate-and-fire (IF) neurons with 
membrane potentials modeled by 



TiVi 



-(«« - E Lti ) + rp( Vi ) + Ei + sJofr£i{t) + /,•(*) + Vi (t). (1) 

Here Ei represents the mean synaptic input current from parts of the system not explicitly modeled. 
A spike-generating current ifi(vi) may be included to emulate the rapid onset of action potentials. 
Unless otherwise specified, we utilize the exponential IF model (EIF), so that ip(v) = A^exp[(t; — 
vt)/&t] |26|. Cells are subject to internally induced fluctuations due to channel noise |37|, and 
externally induced fluctuations due to inputs not explicitly modelled 38 . We model both by 



independent, Gaussian, white noise processes, W cr?Tj£j(i) 39 . An external signal to cell i is 
represented by T]i{t). 

Upon reaching a threshold t>th, an action potential is generated, and the membrane potential is 
reset to v T , where it is held constant for an absolute refractory period r re f. The output of cell i is 
characterized by the times, tik, at which its membrane potential reaches threshold, resulting in an 
output spike train yi(t) = 5{t — U^). Synaptic interactions are modeled by delayed a-functions 



W« ( I exp 



t—TD- 



fi(t) = J2( J ij * where J«(*) = \ V T h J ^ L ^ J 1 ^ TD >i . (2 ) 

3 [0 t< T D,j 

The N x N matrix J contains the synaptic kernels, while the matrix W contains the synaptic 
weights, and hence defines the network architecture. In particular, Wjj = represents the absence 
of a synaptic connection from cell j to cell i. 

Table [T] provides an overview of all parameters and variables. 
Measures of spike time correlation 

We quantify dependencies between the responses of cells in the network using the spike train auto- 
and cross-correlation functions [40] . For a pair of spike trains, yi(t),yj{t), the cross-correlation 
function Cy(r) is defined as 

Cy(r) = cov (yi(t + t), yj(t)) . 

The auto-correlation function Ca(t) is the cross-correlation between a spike train and itself, and 
C(t) is the matrix of cross-correlation functions. Denoting by N yi (ti,t2) = yi(s)ds the number 
of spikes over a time window [ii, ^2]? the spike count correlation, p^-(T), over windows of length r 
is defined as, 

= cov (N yi (t,t + T),N yj (t,t + T)) 
'var (N yt (t, t + T)) var (N yj (t, t + T)) 
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We assume stationarity of the spiking processes (that is, the network has reached a steady state) 
so that Pij(T) does not depend on t. We also use the total correlation coefficient Pj„-(oo) = 
liniT^oo (T) to characterize dependencies between the processes m and yj over arbitrarily long 
timescales. 



The spike count covariance is related to the cross-correlation function by (7,41 
(N yi (t,t + r),N yj (t,t + T)) = [ Cy(s)(r- |a|)da. 



cov 

We can interpret the cross-correlation as the conditional probability that cell j spikes at time 
t + r given that cell i spiked at time t. The conditional firing rate, 

Hij (r) = lim -^Pr (N y . (t + r, t + r + At) > | N yi {t, t + At) > 0) , 

is the firing rate of cell j conditioned on a spike in cell i at r units of time in the past, and 
c ij( T ) = ^(Hy(r) - 7-j). 

Linear response of individual cells 

Neuronal network models are typically described by a complex system of coupled nonlinear stochas- 
tic differential equations. Their behavior is therefore difficult to analyze directly. We will use linear 



response theory 29 35,36,40 to approximate the cross-correlations between the outputs of neurons 
in a network. We first review the linear approximation to the response of a single cell. We illustrate 
the approach using current-based IF neurons, and explain how it can be generalized to other models 
in the Discussion. 

The membrane potential of an IF neuron receiving input eX(t), with vanishing temporal average, 
{X(t)) = 0, evolves according to 

tv = -{v - E L ) + ip(v) + E + V^V^(t) + eX{t). (3) 

The time-dependent firing rate, r(t), is determined by averaging the resulting spike train, y(t) = 
5(t — tj), across different realizations of noise, £(i), for fixed X(t). Using linear response theory, 
we can approximate the firing rate by 

r(t) = r + (A*eX)(t), (4) 

where ro is the (stationary) firing rate when e = 0. The linear response kernel, A(t), characterizes 
the firing rate response to first order in e. A rescaling of the function A(t) gives the spike-triggered 
average of the cell, to first order in input strength, and is hence equivalent to the optimal Weiner 
kernel in the presence of the signal £(i). (40,42 . In Fig. [TJ we compare the approximate firing rate 



obtained from Eq. Q to that obtained numerically from Monte Carlo simulations. 

The linear response kernel A(t) depends implicitly on model parameters, but is independent 
of the input signal, eX(t), when e is small relative to the noise V a 2 r^{t). In particular, A{t) 
is sensitive to the value of the mean input current, E. We emphasize that the presence of the 
background noise, £, in Eq. ([3]) is essential to the theory, as noise linearizes the transfer function 
that maps input to output. 
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Linear response in recurrent networks 

The linear response kernel can be used to approximate the response of a cell to an external input. 
However, the situation is more complicated in a network where a neuron can affect its own activity 
through recurrent connections. To extend the linear response approximation to networks we follow 
the approach introduced by |27| . Instead of using the linear response kernel to approximate the 
firing rate of a cell, we use it to approximate a realization of its output 

y(t)^y 1 (t) = y°(t) + (A*X)(t). (5) 

Here y°(t) represents a realization of the spike train generated by an integrate-and-fire neuron 
obeying Eq. @ with X(t) = 0. 

The central assumption we make is that a cell acts approximately as a linear filter of its inputs. 
Note that Eq. ^ defines a mixed point and continuous process, but averaging y(t) in Eq. ([5j over 
realizations of y gives Eq. Q. Hence, Eq. ([§ can be viewed as a natural generalization of Eq. Q 
where the unperturbed output of the cell is represented as a point process, y°(t), instead of the 
firing rate, tq. 

We first use Eq. ^ to describe spontaneously evolving networks where rjiit) = 0. Equation ([!]) 
can then be rewritten as 

UVi = -(vi- E Lyi ) + + E[ + y[afr£i{t) + (/i(t) - E[/i]), (6) 

where E[ = Ei + E[/j] and E[-] represents the temporal average. 

As a first approximation of the spiking output of cells in the coupled network, we start with 
realizations of spike trains, y®, generated by IF neurons obeying Eq. ^ with fi(t) = E[/j]. This is 
equivalent to considering neurons isolated from the network, with adjusted DC inputs (due to mean 
network interactions). Following the approximation given by Eq. ([5]), we use a. frozen realization of 
all y\ to find a correction to the output of each cell, with X(t) set to the mean-adjusted synaptic 
input, 

X(t) = fi(t) - E[fi] . 

As noted previously, the linear response kernel is sensitive to changes in the mean input current. It 
is therefore important to include the average synaptic input E[/j] in the definition of the effective 
mean input, E[. 

The input from cell j to cell i is filtered by the synaptic kernel Jjj(i). The linear response of 
cell i to a spike in cell j is therefore captured by the interaction kernel Ky defined by 

Ky(t) = (A i *3 ij )(t). 

The output of cell i in response to mean-adjusted input, y®{t) — rj, from cell j can be approximated 
to first order in input strength using the linear response correction 

yl(t) = y°(t)+Y / (Kii*[y']-r j W)- (7) 

3 

We explain how to approximate the stationary rates, rj, in the Methods section. 

The cross-correlation between the processes yj(t) in Eq. Q gives a first approximation to the 
cross-correlation function between the cells (See Methods), 

djir) « Cjj(r) = B[(y}(t + r) - n)(y}(t) - rj)] 

= ayC&(r) + (Kjj * C°jj)(T) + (K 7i * GS)(r) + ^(K ifc * Kj k * C° kk )(r), ^ 

k 
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where we used f~(t) = f(—t). 25 obtained an approximation closely related to Eq. ([8]). They first 
obtained the cross-correlation between a pair of neurons which either receive a common input or 
share a monosynaptic connection. This can be done using Eq. Q, without the need to introduce 
the mixed process given in Eq. |25| then implicitly assumed that the correlations not due to 
one of these two submotifs could be disregarded. The correlation between pairs of cells which were 
mutually coupled (or were unidirectionally coupled with common input) was approximated by the 
sum of correlations introduced by each submotif individually. 

Equation Q provides a first approximation to the joint spiking statistics of cells in a recurrent 
network. However, it captures only the effects of direct synaptic connections, represented by the 
second and third terms, and common input, represented by the last term in Eq. Q. The impact 
of larger network structures, such as loops and chains are not captured, although they may signifi- 
cantly impact cross-correlations |43|-l45|. Experimental studies have also shown that local cortical 



connectivity may not be fully random 46-48 . It is therefore important to understand the effects 
on network architecture on correlations. 

To capture the impact of the full network structure, we propose an iterative approach which 



accounts for successively larger connectivity patterns in the network 33,34]. We again start with 
Vi(t), a realization of a single spike train in isolation. Successive approximations to the output of 
cells in a recurrent network are defined by 

y? +1 (t) = yKt) + £(K<, * [y] - rMt), n > 0. (9) 



To compute the correction to the output of a neuron, in the first iteration we assume that 
its inputs come from a collection of isolated cells: When n = 1, Eq. ^ takes into account only 
inputs from immediate neighbors, treating each as disconnected from the rest of the network. The 
corrections in the second iteration are computed using the approximate cell responses obtained 
from the first iteration. Thus, with n = 2, Eq. ^ also accounts for the impact of next nearest 
neighbors. Successive iterations include the impact of directed chains of increasing length: The 
isolated output from an independent collection of neurons is filtered through n stages to produce 
the corrected response (See Fig. [2|) 

Notation is simplified when this iterative construction is recast in matrix forirj^to obtain 
y n+1 (t) = y°(t) + (K*[y n -r})(t) 

^ n > °> ( 10 ) 

= y (t) + £(K( fc )*[y°-r])(t), " 
k=l 

where y n (t) = [yf (t)] and r = [ri\ are length N column vectors, and K( fc ) represents a fc-fold matrix 
convolution of K with itself. 

1 Let X(t) = [Xij(t)] and Y(t) = [Yy(t)] be rii x ri2 and ri2 x ris matrices of functions, respectively. We define the 
convolution of matrices (X * Y)(i) to be the n\ x 723 matrix of functions with entries defined by 

(X*Y)y(t) = ^(X ifc *y,y)(t). 
k 

Expectations and convolutions commute for matrix convolutions as matrix expectations are taken entry-wise. Each 
entry of a matrix convolution is a linear combination of scalar convolutions which commute with expectations. 
Additionally, we adopt the convention that the zeroth power of the interaction matrix, K°j(t), is the diagonal matrix 
with K°, (t) = 8(t) when i = j. Hence J£°Ai) acts as the identity matrix under matrix convolution. 
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The n approximation to the matrix of cross-correlations can be written in terms of the inter- 

iji 

Tl 



action kernels, Ky, and the autocorrelations of the base processes y° as (See Methods) 



Q,(r) » C» = E[(y»(t + r) - r)(y»(t) - r) 

(11) 



n 

^(K( fc )*C°*(K-)^))(r), n>0, 



where K (t) = K(— t), x( fcT ) = (X( fc )) T , and X( fc ) is the fc-fold matrix convolution of X with itself. 

If we apply the Fourier transform, f(u) = F[f(t)\{uj) = J 00 ^ f(t)e~ 2 ' Klcjt dt, to Eq. (11), we find 
that for each u, 



C» =E[f( W )r( W )] = £ K fc ( W )E[y°(a;)y *(a;)](K 

,/=o 

' n \ 

£k*M E[y°(a,)y° 



(12) 




\fe=o / 

where X* denotes the conjugate transpose of X. The zero-mean Fourier transforms yf of the 
spiking processes yf is denned by yf = T\yf — r^], and / = T(f) for all other quantities. 

For a suitable matrix norm 1 1 • 1 1, when | |K| | < 1, we can take the limit n — > oo in Eq. ( 12 ) 49 , 
to obtain an approximation to the full array of cross-spectra 

C(w) » C°°H = lim C"(w) = (I - KCw))- 1 ^! ^)^ - KVw)) -1 . (13) 

n— >oo 

This equation can also be obtained by generalizing the approach of [27] (also see [13]). In the 



limit n — > oo, directed paths of arbitrary length contribute to the approximation. Equation (13) 
therefore takes into account the full recurrent structure of the network. We will use the spectral 
norm || • H2, and assume that in the networks we study ||K||2 < 1. This condition is confirmed 
numerically when we use Eq. (13). 

Finally, consider the network response to external signals, T]i(t), with zero mean and finite 
variance. The response of the neurons in the recurrent network can be approximated iteratively by 

y n+1 = y° + K * [y" - r] + A * rj, 

where A = diag(^4j) and rj(t) = [t]i(t)]. External signals and recurrent synaptic inputs are both 
linearly filtered to approximate a cell's response, consistent with a generalization of Eq. Q. As in 
Eq. (11), the n th approximation to the matrix of correlations is 

n n— 1 

C(r) « C n (r) = ( kW * C ° * ( K ~) m )( T ) + Yl * CV * ( A_ ) ( ' T) )( r )> 

k,l=0 k,l=0 

where C(t) = E[r){t + r)r](t) T ] is the covariance matrix of the external signals. We can again 
take the Fourier transform and the limit n — > 00, and solve for C(u). If ||K|| < 1, 

C°°(w) = (I - K(w))~ 1 (C°(u;) + A(w)C J? (a;)A*(w))(I - K*{u))-\ (14) 

When the signals comprising r\ are white (and possibly correlated) corrections must be made to 



account for the change in spectrum and response properties of the isolated cells [27,50 51] (See 
Methods). 
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We note that Eq. (10), which is the basis of our iterative approach, provides an approximation 



to the network's output which is of higher than first order in connection strength. This may seem 
at odds with a theory that provides a linear correction to a cell's response, cf. Eq. Q. However, 
Eq. ( 10 ) does not capture nonlinear corrections to the response of individual cells, as the output of 



each cell is determined linearly from its input. It is the input that can contain terms of any order 
in connection strength stemming from directed paths of different lengths through the network. 

Results 



We use the theoretical framework developed above to analyze the statistical structure of the spiking 
activity in a network of IF neurons described by Eq. ([!]) . We first show that the cross-correlation 
functions between cells in two small networks can be studied in terms of contributions from directed 
paths through the network. We use a similar approach to understand the structure of correlations 
in larger all-to-all and random networks. We show that in networks where inhibition and exci- 
tation are tuned for exact balance, only local interactions contribute to correlations. When such 
balance is broken by a relative elevation of inhibition, the result may be increased synchrony in the 
network. The theory also allows us to obtain averages of cross-correlation functions conditioned on 
connectivity between pairs of cells in random networks. Such averages can provide a tractable yet 
accurate description of the joint statistics of spiking in these networks. 

The correlation structure is determined by the response properties of cells together with synaptic 
dynamics and network architecture. Network interactions are described by the matrix of synaptic 
filters, J, given in Eq. (J2j), while the response of cell i to an input is approximated using its linear 
response kernel Aj. Synaptic dynamics, architecture, and cell responses are all combined in the 
matrix K, where Ky describes the response of cell i to an input from cell j (See Eq. Q). The 



correlation structure of network activity is approximated in Eq. ( 13 ) using the Fourier transforms 
of the interaction matrix, K, and the matrix of unperturbed autocorrelations C . 



Statistics of the response of microcircuits 

We first consider a pair of simple microcircuits to highlight some of the features of the theory. We 
start with the three cell model of feed-forward inhibition shown in Fig. |3j^ |52|. The interaction 
matrix, K(w), has the form 



KM 



' ^ 

K E2Ei (uj) K E2 i{oj] 
\K 1Ei {lo) j 



where cells are indexed in the order E\,E2,I. To simplify notation, we omit the dependence of 
K(w) and other spectral quantities on u. 
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Note that K is nilpotent, and the inverse of (I — K) may be expressed as 



(I-K)" 1 = (I + K + K 2 



\ 



1 

Ke 2 e 1 + Ke 2 iKie 1 1 Ke 2 i 



\ 



K IEl 







1 



(15) 



/ 



Substituting Eq. (15) into Eq. (13) yields an approximation to the matrix of cross-spectra. For 
instance, 



c e 2 i = Kie 2 Cj + Ke 2 e 1 Kj Ei C Ei + Ke 2 i\Kie 1 \ 2 C Ei 
= (Ae 2 Je 2 i)Ci + (Ae 2 Je 2 e 1 )(AiJie 1 )*C Ei 



(16) 



+ (Ae 2 Je 2 e 1 )\A i J IEi \ C El . 



hi 



Figure [3^3 shows that these approximations closely match numerically obtained cross-correlations. 
C\ is the uncoupled power spectrum for cell X. 



Equation (16) gives insight into how the joint response of cells in this circuit is shaped by the 



features of the network. The three terms in Eq. ( 16 ) are directly related to the architecture of the 



microcircuit: Term I represents the correlating effect of the direct input to cell E^ from cell I. Term 
II captures the effect of the common input from cell E% . Finally, term III represents the interaction 
of the indirect input from E\ to E% through I with the input from E\ to I (See Fig. [3]C). A change 
in any single parameter may affect multiple terms. However, the individual contributions of all 
three terms are apparent. 

To illustrate the impact of synaptic properties on the cross-correlation between cells E2 and I 
we varied the inhibitory time constant, 77 (See Fig. [3^3 and C). Such a change is primarily reflected 
in the shape of the first order term, I: Multiplication by Je 2 i is equivalent to convolution with the 
inhibitory synaptic filter, J E2 j. The shape of this filter is determined by tj (See Eq. Q), and a 
shorter time constant leads to a tighter timing dependency between the spikes of the two cells |25[ 
53-56 . In particular, Ostojic et al. made similar observations using a related approximation. In 
the FFI circuit, the first and second order terms, I and II, are dominant (red and dark orange, 
Fig. [3^3). The relative magnitude of the third order term, III (light orange, Fig. [3^3), is small. The 
next example shows that even in a simple recurrent circuit, terms of order higher than two may be 
significant. 

More generally, the interaction matrices, K, of recurrent networks are not nilpotent. Consider 
two reciprocally coupled excitatory cells, E\ and E% (See Fig. [IK, left). In this case, 



K 



/ ~ \ 

Ke 1 e 2 



\Ke 2 e x 







so that 



(I-K)- 1 



1 — Ke 1 e 2 Ke 2 e 1 



(I + K) 
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Equation ( 13 ) gives the following approximation to the matrix of cross-spectra 



1 

|1 — Ke 1 e 2 Ke 2 e 1 \ 



1 

|1 — Ke 1 e 2 Ke 2 e 1 \ 



;(I + K) 



\ 



Gkj 



(I + K*) 



/ C Ei + \K Ei e 2 \ 2 C% 2 K* E2Ei C Ei +K Ei e 2 C e ) 



(17) 



Ke 2 E 1 C% 1 + K E X E 2 C E 2 C E 2 + \ K E 2 E 1 \ 2 C% 1 



J 



In contrast to the previous example, this approximation does not terminate at finite order in 
interaction strength. After expanding, the cross-spectrum between cells E\ and E<i is approximated 
by 



Ly E 1 E 2 



^2 ^E 1 E 2 KE 2 E 1 ) k {K El E 2 K% 2 E 1 ) l (KE 2 E 1 CE 1 + K El E 2 C% 2 ) 
k,l=0 



(18) 



Directed paths beginning at E\ and ending at E2 (or vice-versa) are of odd length. Hence, this 
approximation contains only odd powers of the kernels K Ei Ei each corresponding to a directed 
path from one cell to the other. Likewise, the approximate power spectra contain only even powers 
of the kernels corresponding to directed paths that connect a cell to itself (See Fig. (4jA). 

The contributions of different sub-motifs to the cross- and auto-correlations are shown in 
Figs. |4p, D when the isolated cells are in a near-threshold excitable state (CV ~ 0.98). The 
auto-correlations are significantly affected by network interactions. We also note that chains of 
length two and three (the second and third submotifs in Fig. |4]A.) provide significant contributions. 
Earlier approximations do not capture such corrections |25| . 

The operating point of a cell is set by its parameters {ti,El,u etc.) an d the statistics of its 
input (Ei,ai). A change in operating point can significantly change a cell's response to an input. 
Using linear response theory, these changes are reflected in the response functions Aj, and the 
power spectra of the isolated cells, C°. To highlight the role that the operating point plays in the 



approximation of the correlation structure given by Eq. (13), we elevated the mean and decreased 
the variance of background noise by increasing E{ and decreasing <7j in Eq. Q. With the chosen 
parameters the isolated cells are in a super-threshold, low noise regime and fire nearly periodically 
(CV « 0.31). After the cells are coupled, this oscillatory behavior is reflected in the cross- and auto- 
correlations where the dominant contributions are due to first and zeroth order terms, respectively 
(See Figs.[4t\G). 



Orders of coupling interactions: It is often useful to expand Eq. (13) in terms of powers of 
K [32]. The term K"C°(K*) m in the expansion is said to be of order re + m. Equivalently, in the 
expansion of CJy , the order of a term refers to the sum of the powers of all constituent interaction 
kernels K a ^. We can also associate a particular connectivity submotif with each term. In particular, 
n th order terms of the form 

ic ic . . .ic r 10 

are associated with a directed path j a\ —)■■•• — )■ a n _2 — > a n _i — > i from cell j to cell i. 
Similarly, the term C^K* 01 • • • K* n _ 2an _ 1 K* n _ • corresponds to a n-step path from cell i to cell j. 
An (re + rre) th order term of the form 

■"aoao^aobi ' ' ' iv 6 m _2fe m -l iV bm-:U 



x? x? . . . K" P° K"* . . . K"* k 

■ rs -«a n _l j:v a rl _ia n _2 ^aiao ^a a -"^aobi xv 6 m _ 2 bm-i b 



10 



represents the effects of an indirect common input n steps removed from cell i and m steps removed 
from cell j. This corresponds to a submotif of the form i <— a n -\ ao — > b\ —>•••• — )• fe n _i — > 

j consisting of two branches originating at cell ao- (See Fig. [5j and also Fig. [6]A and the discussion 
around Eqs. (16|18).) 



Statistics of the response of large networks 

The full power of the present approach becomes evident when analyzing the activity of larger 
networks. We again illustrate the theory using several examples. In networks where inhibition 
and excitation are tuned to be precisely balanced, the theory shows that only local interactions 
contribute to correlations. When this balance is broken, terms corresponding to longer paths 
through the network shape the cross-correlation functions. One consequence is that a relative 
increase in inhibition can lead to elevated network synchrony. We also show how to obtain tractable 
and accurate approximation of the average correlation structure in random networks. 



A symmetric, all— to— all network of excitatory and inhibitory neurons We begin with 
an all-to-all coupled network of ./V identical cells. Of these cells, Ne make excitatory, and Nj 
make inhibitory synaptic connections. The excitatory cells are assigned indices 1, . . . , Ne, and the 
inhibitory cells indices Ne + 1, • • ■ , N. All excitatory (inhibitory) synapses have weight We = 

{Wj 



jyj), and timescale te (tj). The interaction matrix K may then be written in block form, 



K = AJ, 



where 



^Je1n i n e Ji1n i n I j 



Here ljViiVa is the N% x N2 matrix of ones, Jx is the weighted synaptic kernel for cells of class 
X (assumed identical within each class), and A is the susceptibility function for each cell in the 
network. Although the effect of autaptic connections (those from a cell to itself) is negligible (See 
SI Fig. 2, their inclusion significantly simplifies the resulting expressions. 

We define JIe = NeJe, P<i = N1J1, and ft = £ie + fii- Using induction, we can show that 

K k = A k jl k - 1 3. 

Direct matrix multiplication yields 

JJ* = ft c l NN where /2 C = N E \ J E ? + iV}| J/| 2 , 
which allows us to calculate the powers K fc K z * when k,l^0, 

K k K l * = A k (A*) l fl k - 1 (ii*) l - 1 fl c l NN . 



An application of Eq. (13) then gives an approximation to the matrix of cross-spectra: 

2 



C°° = C° K fc K** = C 



k,l=0 




J + 




J* + 



A 

1-AfL 



Acljviv + Tv 



(19) 



11 



The cross-spectrum between two cells in the network is therefore given by 



r/~iooi /^O 



A 



fly 



+ 



A 



1 — Ajx I Ny \1-Ap, I N x 



+ 



A 



1-AJi 



Ac + $ 



(20) 



where X £ {E, I}. In Eq. (20) the first two terms represent the effects of all unidirectional chains 



originating at cell j and terminating at cell i, and vice versa. The third term represents the effects 
of direct and indirect common inputs to the two neurons. In Fig. [6]A, we highlight a few of these 
contributing motifs. 

Interestingly, when excitation and inhibition are tuned for precise balance (so that A = As+A-f = 



0), Eq. (20) reduces to 



r/~iocn /^O 



A^+A*^ 



VY 
~Ny 



N x 



+ |i| 2 A c 



(21) 



Effects of direct connections between the cells are captured by the first two terms, while those of 
direct common inputs to the pair are captured by the third term. Contributions from other paths 
do not appear. In other words, in the precisely balanced case only local interactions contribute to 
correlations. 

To understand this cancelation intuitively, consider the contribution of directed chains originat- 
ing at a given excitatory neuron, j. For r > 0, the cross-correlation function, Cjj(r), is determined 
by the change in firing rate of cell i at time r given a spike in cell j at time 0. By the symme- 
try of the all-to-all connectivity and stationarity, the firing of cell j has an equal probability of 
eliciting a spike in any excitatory or inhibitory cell in the network. Due to the precise synaptic 
balance, the postsynaptic current generated by the elicited spikes in the excitatory population will 
cancel the postsynaptic current due to elicited spikes in the inhibitory population on average. The 
contribution of other motifs cancel in a similar way. 

In Fig. [6)3, we show the impact of breaking this excitatory-inhibitory balance on cross-correlation 
functions. We increased the strength and speed of the inhibitory synapses relative to excitatory 
synapses, while holding constant, for sake of comparison, the long window correlation coefficients, 
p(oo). at ~ 0.05 Moreover, the degree of network synchrony, characterized by the short window 
correlation coefficients, is increased (See Fig.[6^> inset). Intuitively, a spike in one of the excitatory 
cells transiently increases the likelihood of spiking in all other cells in the network. Since inhibition 
in the network is stronger and faster than excitation, these additional spikes will transiently decrease 
the likelihood of spiking in twice removed cells. 

Linear response theory allows us to confirm this heuristic observation, and quantify the impact 



of the imbalance on second order statistics. Expanding Eq. (20) for two excitatory cells to second 
order in coupling strength, we find 



A W E +A W E + ^ + A ^W E ^ A ' 



I.I 



+ 0(\\K\ 



(22) 



The complete cancellation between contributions of chains involving excitatory and inhibitory cells 
no longer takes place, and the two underlined terms appear as a consequence (compare with 
Eq. (21)). These underlined terms capture the effects of all length two chains between cells 



or Ej, starting at one and terminating at the other. The relative strengthening of inhibition im- 
plies that chains of length two provide a negative contribution to the cross-correlation function 
at short times (c/. |57|). Additionally, the impact of direct common input to cells Ei and Ej on 
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correlations is both larger in magnitude (because we increased the strength of both connection 
types) and sharper (the faster inhibitory time constant means common inhibitory inputs induces 



sharper correlations). These changes are reflected in the second order term |A| 2 /2 C in Eq. (22). 

In sum, unbalancing excitatory and inhibitory connections via stronger, faster inhibitory synapses 
enhances synchrony, moving a greater proportion of the covariance mass closer to r = (See 
Fig. [6^3). To illustrate this effect in terms of underlying connectivity motifs, we show the contribu- 
tions of length two chains and common input in both the precisely tuned and non-precisely tuned 
cases in Fig. [6p. A similar approach would allow us to understand the impact of a wide range of 
changes in cellular or synaptic dynamics on the structure of correlations across networks. 



Random, fixed in-degree networks of homogeneous excitatory and inhibitory neurons 

Connectivity in cortical neuronal networks is typically sparse, and connection probabilities can 
follow distinct rules depending on area and layer |58| . The present theory allows us to consider 
arbitrary architectures, as we now illustrate. 

We consider a randomly connected network of Ne excitatory and Nj inhibitory cells coupled 
with probability p. To simplify the analysis, every cell receives exactly pNe excitatory and pNj 
inhibitory inputs. Thus, having fixed in-degree, each cell receives an identical level of mean synaptic 
input. In addition, we continue to assume that cells are identical. Therefore, the response of each 
cell in the network is described by the same linear response kernel. The excitatory and inhibitory 
connection strengths are Ge/{pNe) an d Gi/(pNj), respectively. The timescales of excitation and 
inhibition may differ, but are again identical for cells within each class. 

The approximation of network correlations (Eq. (|13[)) depends on the realization of the con- 
nectivity matrix. For a fixed realization, the underlying equations can be solved numerically to 
approximate the correlation structure (See Fig. [Tj^). However, the cross-correlation between a pair 
of cells of given types has a form which is easy to analyze when only leading order terms in 1/N 
are retained. 

Specifically, the average cross-spectrum for two cells of given types is (See SI Section 1) 



E^C-} 



.4 



fly 



+ 



A 



1 — A[i ) Ny \1-Aftl N x 



+ 



A 



1 - Afl 



+ 0(1/N 2 



(23) 



when i ^ j. This shows that, to leading order in 1/N, the mean cross-spectrum between two cells in 



given classes equals that in the all-to-all network (see Eq. (20)). Therefore our previous discussion 
relating network architecture to the shape of cross-correlations in the all-to-all network extends to 
the average correlation structure in the random network for large N. 

32 1 derived similar expressions for the correlation functions in networks of interacting Hawkes 



processes 59,60 by assuming either the network is regular (i.e., both in- and out-degrees are fixed) 
or has a sufficiently narrow degree distribution. Our analysis depends on having fixed in-degrees, 
and we do not assume that networks are fully regular. Both approaches lead to results that hold 
approximately (for large enough N) when the in-degree is not fixed. 



Average correlations between cells in the random network conditioned on first order 
connectivity As Fig. [7T3 shows there is large variability around the mean excitatory-inhibitory 



cross-correlation function given by the leading order term of Eq. (23 ). Therefore, understanding the 



average cross-correlation between cells of given types does not necessarily provide much insight into 
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the mechanisms that shape correlations on the level of individual cell pairs. Instead, we examine 
the average correlation between a pair of cells conditioned on their first order (direct) connectivity. 

We derive expressions for first order conditional averages correct to 0(1/N 2 ) (See SI Sec. 2). 
The average cross-spectrum for a pair of cells with indices i ^ j, conditioned on the value of the 
direct connections between them is 



E-| C5?| Jij, Jjo > — 
J J J J ieX,j£Y 



A J ij ~t~ A J a ~t~ 



A z pL \ fly 



+ 



A 2 



I'- 



1-Afi Ny \l-Aji N x 



A 



l-Afl 



He 



(24) 



+ 0(l/iV 2 ). 



Here we set Jij = if we condition on the absence of a connection j 
condition on its presence. The term J,j is set similarly. 



and Jij = Jy/p if we 



Although Eq. (24) appears significantly more complicated than the cell- type averages given 
in Eq. (23), they only differ in the underlined, first order terms. The magnitude of expected 



contributions from all higher order motifs is unchanged and coincides with those in the all-to-all 
network. 

Figure [7p shows the mean cross-correlation function for mutually coupled excitatory-inhibitory 
pairs. Taking into account the mutual coupling significantly reduces variability (Compare with 
Fig.[7f3). To quantify this reduction, we calculate the mean reduction in variability when correlation 
functions are computed conditioned on the connectivity between the cells. For a single network, 
the relative decrease in variability can be quantified usingP] 



Me 



1 



£ 

(M)6T 
i>j 



C£T( 7 



where T represents pairs of cells of a given type and connection (in the present example these are 
reciprocally coupled excitatory- inhibitory pairs), Nt is the number of pairs of that type in the 
network, C^ T (r) is the leading order approximation of average correlations given only the type of 
cells in T (as in Eq. (23)), and C|i oc (r) the leading order approximation to average correlations 



conditioned on the first order connectivity of class T (as in Eq. (24)). Figure [7p shows fi e 
averaged over twenty networks. In particular, compare the reduction in variability when condi- 
tioning on bidirectional coupling between excitatory-inhibitory pairs shown in Figs.[7^B,C, with the 
corresponding relative error in Fig. [7£) (circled in red). 



Discussion 

We have developed a general theoretical framework that can be used to describe the correlation 
structure in a network of spiking cells. This theory allows us to find tractable approximations of 
cross-correlation functions in terms of the network architecture and single cell response properties. 
The approach was originally used to study the population response of the electrosensory lateral line 



lobe of weakly electric fish 27 . The key approximation relies on the assumption that the activity of 



cells in the network can be represented by a mixed point and continuous stochastic process, as given 
2 We make use of the norm || ■ || 2 defined by ||/|| 2 = (/ |./f) 1/2 . 
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in Eq. ^ . An iterative construction then leads to the expressions for approximate cross-correlations 
between pairs of cells given by Eq. (13). 



[25] obtained formulas for cross-correlations that correspond to the first step in this iterative 
construction, given in Eq. Their approach captures corrections due to direct coupling (first 
order terms) and direct common input (second order terms involving second powers of interaction 
kernels). Our approach can be viewed as a generalization that also accounts for length two directed 
chains, along with all higher order corrections. As Fig. [4] illustrates, these additional terms can yield 
significant contributions to the structure of cross-correlations. We note that another distinction 
between our approach and that of 25 is that the present approach also allows us to calculate 
corrected auto-correlations, whereas the framework of 25 did not allow for adjustments to auto- 
correlations. 



[32] analyzed the correlation structure in networks of interacting Hawkes processes [59|60| using 
an approach similar to the one presented here. In both cases, we represented correlations between 
cell pairs in terms of contributions of different connectivity motifs. However, our methods differ 
in important ways: while their expressions are exact for Hawkes processes, 32 did not attempt 
to match their results to more physiological cell models, and did not account for the response 
properties of individual cells, although it may be possible to approximately fit the Hawkes models 
to such models. [32] examined only total spike count covariances, which equal the integrals of the 
cross-correlation functions. This leads to a loss of information about the temporal structure of 
correlations. However, as they note, their approach can be extended to obtain complete cross- 
correlation functions for the Hawkes model. 

To illustrate the power of the theory in analyzing the factors that shape correlations, we con- 
sidered a number of simple examples for which the approximation Eq. ( 13 ) is tractable. We showed 
how the theory can be used both to obtain intuition for the effects that shape correlations, and to 
quantify their impact. In particular, we explained how only local connections affect correlations 
in a precisely tuned all-to-all network, and how strengthening inhibition may synchronize spiking 
activity. 

Linear response methods are perturbative. For Eq. ^ to be valid, neurons need to respond 
approximately linearly to their inputs. This will only be true if inputs are "weak" relative to the 
dynamical operating point of the cell. We assume the presence of a white noise background which 
linearizes the transfer function, although it is possible to extend the present methods to colored 



background noise 26,61 



It may be surprising that an approach based on linear response theory can provide corrections 
to cross-correlations of arbitrary order in network connectivity. Corrections to firing activity which 
are higher order in network connectivity emerge from the linear correction in Eq. (10). A full 
expansion of firing activity would include terms arising from corrections to the input-output transfer 
of the individual cells beyond those captured by the linear response approximation. Formally, 
including such terms would be necessary to capture all contributions of a given order in network 
connectivity [33j|34]. However, the high accuracy demonstrated by our method indicates that, at 
least in some cases, these additional terms are quite small. In short, our approximation neglects 
higher-order corrections to the input-output transfer of individual cells, which is compatible with 
the assumption that the presence of background noise causes this transfer to be close to linear. 

As expected from the preceding discussion, simulations suggest that, for IF neurons, our ap- 
proximations become less accurate as cells receive progressively stronger inputs. The "physical" 
reasons for this loss of accuracy could be related to interactions between the "hard threshold" and 
incoming synaptic inputs with short timescales. While the theory will work for short synaptic 
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timescales, it will improve for slower synaptic dynamics, limiting towards being essentially exact in 
the limit of arbitrarily long synaptic time constants (Note the improvement in the approximation 
for the FFI circuit for the slower timescale as exhibited in Fig. |3j). However, as we have shown, 
the theory remains applicable in a wide range of dynamical regimes, including relatively low noise, 
superthreshold regimes where cells exhibit strong oscillatory behavior. Moreover, the theory can 
yield accurate approximations of strong correlations due to coupling: for the bidirectionally coupled 
excitatory circuit of Fig. |4j the approximate cross-correlations match numerically obtained results 
even when correlation coefficients are large (Pe 1 e 2 ( 00 ) ~ 0.8 i n t ne excitable regime, ~ 0.5 in the 
oscillatory regime). 

Although we have demonstrated the theory using networks of integrate-and-fire neurons, the 
approach is widely applicable. The linear response kernel and power spectrum for a general inte- 
grate and fire neuron model can be easily obtained [30]. In addition, it is also possible to obtain 
the rate, spectrum, and rate response function for modulation of the mean conductance in the 
case of conductance-based (rather than current-based) synapses (See 62 and SI Sec. 3). As the 



linear response kernel is directly related to the spike triggered average 25 31 , the proposed the- 
oretical framework should be applicable even to actual neurons whose responses are characterized 
experimentally. 

The possibilities for future applications are numerous. For example, one open question is how 
well the theory can predict correlations in the presence of adaptive currents [62 1. In addition, the 
description of correlations in terms of architecture and response properties suggests the possibility 
of addressing the difficult inverse problem of inferring architectural properties from correlations 1 23 — 



25 63 . Overall, it is our hope that the present approach will prove a valuable tool in moving the 
computational neuroscience community towards a more complete understanding of the origin and 
impact of correlated activity in neuronal populations. 



Methods 

Numerical methods Simulations were run in C+- 1-, and the stochastic differential equations 
were integrated with a standard Euler method with a time-step of 0.01ms. General parameter 
values were as follows: T{ = 20ms, Eea + E{ = — 54mV, cij = \/l2mV, Vth = 20mV, v r = — 54mV, 
r re f = 2ms, Vt = — 52.5mV, = 1.4mV, te = 10ms, tj = 5ms, td,{ = 1ms. Marginal statistics 
(firing rates, uncoupled power spectra and response functions) were obtained using the threshold 
integration method of [30] in MATLAB. All code is available upon request. 

Calculation of stationary rates in a recurrent network The stationary firing rate of an IF 
neuron can be computed as a function of the mean and intensity of internal noise (Ei,ai) and other 
cellular parameters (Tj,£x^, etc..) |64| . Denote the stationary firing rate of cell i in the network 
by rj, and by ri^(E, a) the stationary firing rate in the presence of white noise with mean E and 
variance a 2 . We keep the dependencies on other parameters are implicit. The stationary rates, rj, 
in the recurrent network without external input are determined self-consistently by 

n = r ifl (E / i ,a i ) = r ifl {Ei + } J W i jr j , <7j) i = 1, ...,N , 

j 

where we used E[/j] = Ylj ^ij^iVj] = J2j ^ij r j- This equality holds because the synaptic kernels, 
Jij, were normalized to have area Wjj. These equations can typically be solved by fixed-point 
iteration. 
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Note that this provides an effective mean input, E[, to each cell, but does not give adjustments 
to the variance, o"j. We assume that the major impact of recurrent input is reflected in E^, and 
ignore corrections to the cell response involving higher order statistics of the input. This approach 
is valid as long as fluctuations in the recurrent input to each cell are small compared to o"j, and 



may break down otherwise 28 



Derivation of Eq. ([8]): Equation ^ can be obtained by a direct calculation: 

E[(yj(t + r) - n ){y}{t) - Tj )] = E[(y°(i + r) - r,)(^(t) - rj)] + £ E[(K lk * [y° k - r k ])(t + r)(y°(t) 

k 

+ ^[(^(i + r) - n){K jk * [y° k - r k ])(t)] + £ E[(K lfc * fog - r fc ])(t + r)(Kj, * [yf - n ])( 

fc fc,Z 

= %C°, (r) + (Ktf * C°,)(r) + (K^ * C°J(r) + ^(K, fc * KJ k * Cg fc )(r) 



Verification of Eq. (11): First, Eq. (10) directly implies that 



y n (t) = y°(t) + ]T(K fc * [y° - r])(t), n > 0, 



fc=l 



which we may use to find, for each n > 0, 
C»^E[(y™(t + r)-r)(y"(t)-r) T ] 

n 

= E[(y°(t + r) - r)(y°(t) - rf] + £ e[(K* * [y° - r])(t + r)(y°(t) 



fe=l 



71 71 

+ 5>[(y°(i + r) - r)(K fc * [y° - r]f(i)] + £ e[(K* * [y° - r])(t + r)(K fc * [y° - r]) T (t) 

k=l k,l=l 

n n n 

= C°(r) + ^(K fc * C°)(r) + ^(C° * (KT) fcT )(r) + ^ (K fc * C° * (K-) it )(t). 



k=l 



k=l 



k,l=l 



(25) 



Since K?-(£) = Eq. (|25j) is equivalent to Eq. ([TT]). 



Correction to statistics in the presence of an external white noise signals. Expres- 



sion (14) can be used to compute the statistics of the network response to inputs rji(t) of finite 



variance. As noted by |27| , when inputs have infinite variance additional corrections are necessary. 
As a particular example, consider the case where the processes are correlated white noise, i.e., when 
Vi(t) = \fcx c {t) + Vl — cxj(t), where x c ,Xi are independent white noise processes with variance a e . 
Then each rji is also a white noise process with intensity of , but E[r)i(t + r)rjj(t)] = [SijS(r) + (1 — 
8ij)c8(r)]a? . The firing rate of cell i in response to this input is r; t = r^{E'^ (o"i) 2 + (°f ) 2 )> an d 
the point around which the response of the cell is linearized needs to be adjusted. 

Finally, we may apply an additional correction to the linear response approximation of autocor- 



relations. For simplicity, we ignore coupling in Eq. (14) (so that K = 0). Linear response predicts 
that Cjj(cj) = C^wjof) + (af) 2 \Ai(u)\ 2 , where we have introduced explicit dependence on of, 
the variance of white noise being received by an IF neuron with power spectrum C^(u;; af), in the 
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absence of the external signal. The approximation may be improved in this case by making the 
following substitution in Eq. ([li]) [27j[fl] : 



The response function A should be adjusted likewise. 
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Figure 1. Illustrating Eq. Q. (A) The input to the post-synaptic cell is a fixed spike train 
which is convolved with a synaptic kernel. (B) A sample voltage path for the post-synaptic cell 
receiving the input shown in A) in the presence of background noise. (C) Raster plot of 100 
realizations of output spike trains of the post-synaptic cell. (D) The output firing rate, r(t), 
obtained by averaging over realizations of the output spike trains in C). The rate obtained using 
Monte Carlo simulations (shaded in gray) matches predictions of linear response theory obtained 
using Eq. Q (black). 
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Figure 2. Iterative construction of the linear approximation to network activity. (A) 

An example recurrent network. (B)-(D) A sequence of graphs determines the successive 



approximations to the output of neuron 1. Processes defined by the same iteration of Eq. (10) 



have equal color. (B) In the first iteration of Eq. (10), the output of neuron 1 is approximated 



using the unperturbed outputs of its neighbors. (C) In the second iteration the results of the first 
iteration are used to define the inputs to the neuron. For instance, the process y\ depends on the 
base process y\ which represents the unperturbed output of neuron 1. Neuron 4 receives no inputs 
from the rest of the network, and all approximations involve only its unperturbed output, y\. (D) 
Cells 3 and 4 are not part of recurrent paths, anjj^ their contributions to the approximation are 
fixed after the second iteration. However, the recurrent connection between cells 1 and 2 implies 
that subsequent approximations involve contributions of directed chains of increasing length. 
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Figure 3. The relation between correlation structure and response statistics in a 
feed- forward inhibitory microcircuit. (A) The FFI circuit (left) can be decomposed into 
three submotifs. Equation (16) shows that each submotif provides a specific contribution to the 



cross-correlation between cells E2 and /. (B) Comparison of the theoretical prediction with the 
numerically computed cross-correlation between cells E% and /. Results are shown for two 
different values of the inhibitory time constant, tj (tj = 5ms, solid line, tj = 10ms, dashed line). 
(C) The contributions of the different submotifs in panel A are shown for both tj = 5ms (solid) 
and tj = 10ms (dashed). Inset shows the corresponding change in the inhibitory synaptic filter. 
The present color scheme is used in subsequent figures. Connection strengths were ±40 mV • ms 
for excitatory and inhibitory connections. In each case, the long window correlation coefficient 
Pe 2 i between the two cells was ~ —0.18. 
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Figure 4. The relation between correlation structure and response statistics for two 
bidirectionally coupled, excitatory cells. (A) The cross-correlation between the two cells 
can be represented in terms of contributions from an infinite sequence of submotifs (See Eq. (18)). 
Though we show only a few "chain" motifs in one direction, one should note that there will also 
be contributions to the cross-correlation from chain motifs in the reverse direction in addition to 
indirect common input motifs (See the discussion of Fig. [5J. (B), (E) Linear response kernels in 
the excitable (B) and oscillatory (E) regimes. (C), (F) The cross-correlation function with first 
and third order contributions computed using Eq. ( |17[ ) in the excitable (C) and oscillatory (F) 
regimes. (D), (G) The auto-correlation function with zeroth and second order contributions 
computed using Eq. (17) in the excitable (D) and oscillatory (G) regimes. In the oscillatory 
regime, higher order contributions were small relative to first order contributions and are therefore 
not shown. The network's symmetry implies that cross-correlations are symmetric, and we only 
show them for positive times. All cross-correlations are nearly indistinguishable from those 
obtained from simulations (See SI Fig. 3 for comparisons with simulations). Connection strengths 
were 40 mV • ms. The long window correlation coefficient Pe 1 E2(°°) between the two cells was 
~ 0.8 in the excitable regime and ~ 0.5 in the oscillatory regime. The ISI CV was approximately 
0.98 for neurons in the excitable regime and 0.31 for neurons in the oscillatory regime. 
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Figure 5. The motifs giving rise to terms in the expansion of Eq. (13). (A) Terms 
containing only unconjugated (or only conjugated) interaction kernels K. a b correspond to directed 
chains. (B) Terms containing both unconjugated and conjugated interaction kernels K a {, 
correspond to direct or indirect common input motifs. 
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Figure 6. All— to— all networks and the importance of higher order motifs. (A) Some of 
the submotifs contributing to correlations in the all-to-all network. (B) Cross-correlations 
between two excitatory cells in an all-to-all network (Ne = 80, Nj = 20) obtained using Eq. (19) 
(Solid - precisely tuned network with fl = 

[Ge = 175 mV-ms, Gj = —(Ne/Nj)Ge = —700 mV-ms, te = 77 = 10 ms], dashed - non-precisely 
tuned network with p, / [Ge = 210 mV-ms, Gi = —1050 mV-ms, te = 10 ms, tj = 5 ms]). (C) 
Comparison of first and second order contributions to the cross-correlation function in panel A in 
the precisely tuned (left) and non-precisely tuned (right) network. In both cases, the long window 
correlation coefficient p(oo) was fixed at 0.05. 
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Figure 7. Correlations in random, fixed in-degree networks. (A) A comparison of 
numerically obtained excitatory-inhibitory cross-correlations to the approximation given by 



Eq. (24). (B) Mean and standard deviation for the distribution of correlation functions for 
excitatory-inhibitory pairs of cells. (Solid line - mean cross-correlation, shaded area - one 
standard deviation from the mean, calculated using bootstrapping in a single network 
realization). (C) Mean and standard deviation for the distribution of cross-correlation functions 
conditioned on cell type and first order connectivity for a reciprocally coupled 
excitatory-inhibitory pair of cells. (Solid line - mean cross-correlation function, shaded area - one 
standard deviation from the mean found by bootstrapping). (D) Average reduction in L 2 error 
between cross-correlation functions and their respective first-order conditioned averages, relative 
to the error between the cross-correlations and their cell-type averages. Blue circles give results 
for a precisely tuned network, and red squares for a network with stronger, faster inhibition. 
Error bars indicate two standard errors above and below the mean. Ge,Gj,te,ti for panels A-C 
are as in the precisely tuned network of Fig. [6j and the two networks of panel D are as in the 
networks of the same figure. 
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Vi,Ti,E L i,(Ti 



Ei,ai 



Vth,Vr,Tref 
TS,i,1~D,i 



w 



3 tj (t) 
Cij(r) 

N Vt (t,t + T), Pij (T) 
n , Ai (t) , C% 

Kij(t) 

y?(t),cg.(t) 



Membrane potential, membrane time constant, leak reversal potential, and 
noise intensity of cell i. 

Mean and standard deviation of the background noise for cell i. 
Membrane potential threshold, reset, and absolute refractory period for cells. 
Spike generating current, soft threshold and spike shape parameters for the 
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Synaptic input from other cells in the network, and external input to cell i. 
Synaptic time constant and delay for outputs of cell i. 
Spike train of cell i. 

The j — > i synaptic weight, describes the area under a single post-synaptic 
current for current-based synapses. 

The j — > i synaptic kernel - equals the synaptic weight Wjj multiplied times 
the synaptic filter for outputs of cell j. 

The cross-correlation function between cells i,j defined by Cij{r) = 
cov(yi(t + r),yj(t)). 

Spike count for cell i, and spike count correlation coefficient for cells i,j over 
windows of length T. 

Stationary rate, linear response kernel and uncoupled auto-correlation func- 
tion for cell ij. 

The j — > i interaction kernel - describes how the firing activity of cell i is 
perturbed by an input spike from cell j. It is defined by Kjj(i) = (Ai* Jij)(t). 

The n th order approximation of the activity of cell i in a network which 
accounts for directed paths through the network graph up to length n ending 
at cell i, and the cross-correlation between the n th order approximations of 
the activity of cells i,j. 

g(oj) is the Fourier transform of g(t) with the convention 



g(t)dt 



Table 1. Notatioggused in the text. 



